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PREFACE 


\ 

Autonomous 
Inspection  System 


Autonomous  Inspection  System  that  can 
circumnavigate  an  entire  human-rated  spacecraft 

>  Same  concept  for  Hovering  Mini  UAV  and  inspections  of 
aircraft,  buildings,  bridges,  roads,  etc. 

>  May  possibly  be  expanded  with  UUV  for  ship  hull  inspections 

2D  /  3D  Imaging  System  that  can  capture  high 
resolution  5D  images 

Integration  with  COTS  CAD  Systems 


Automated  image  inspection  of  As-ls  and  Baseline  data 
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BACKGROUND 


\ 

Initiative 


Columbia  Accident  Investigation  Report,  Vol.  I, 
August  2003,  Part  One: 

>  R3.3. 1  -  .  implement  a  comprehensive  inspection 

plan...” 

>  R6,4-1  - "...  develop  a  comprehensive  autonomous 
(independent  of  Station)  inspection  ...” 

>  R3.4-3  - "...  capability  to  obtain  and  downlink  high- 
resolution  images  ...” 

>  R1 0.3.1  -  “...  Digitize  the  closeout  photograph  system 
so  that  images  are  immediately  available  for  on-orbit 
troubleshooting.” 

>  R1 0.3.2  -  . .  Converting  all  drawings  to  a  computer- 
aided  drafting  system  ...” 
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AIS  -  AerCAM 


\ 

Previous  Programs 


Miniature  Autonomous  Extravehicular  Robotic 
Camera 


>  Dr.  Steven  E.  Fredrickson:  Automation, 
Robotics,  and  Simulation  Division  /  ER6 
NASA  Johnson  Space  Center 

>  Designed: 


LED  Army 


Single 

Camera 


□  uni 
C  ame  ms 


GPS  Antenna 


Remotely  Piloted  &  Autonomous  Operations 
Equipped  with  2  Video  Imagers 
1  High  Resolution  Still-Frame  Imager 
Dimension:  7.5”  diameter 
Weight:  11  lbs 
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\ 

Previous  Programs 


MANUFACTURING  -  COTS  CAD 


❖  Current  State: 

>  CAD  systems  have  capabilities  to  design  &  model  3D 
components 

>  CAD  systems  have  capabilities  to  detect  “dimensional 
collisions”  during  model  design 


❖  Desired  Enhancements: 

>  Import  feature  of  3D  Mesh  Data 

>  Automated  Live  and  Baseline  Image  data  analytics  and 
reporting 
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TECHNICAL  /  PROG  RAM M A  TIC 


;  \ 

Objectives 


Operational  Concept: 

>  An  Autonomous  System  that  can  detect  external  surface 
changes  (damage)  to  current  and  future  human-rated 
spacecraft  during  flight  using  2D  /  3D  imaging  systems 

>  Decrease  the  physical  inspection  processing  time  by 
systematically  detecting  and  reporting  attribute  changes  of  an 
object 


Technical  Objective: 

>  Develop  a  high  resolution  5D  imaging  module  for  Autonomous 
Inspection  Systems  that  can  detect  and  report  changes  from  a 
baseline  set  of  images  and/or  CAD  data  base 
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TECHNICAL  /  PROGRAMMA  TIC 


;  \ 

Objectives 


Inspection  Accuracy: 

>  Higher  level  of  image  detail  not  easily  seen  by  human  eye 

>  3  Dimensional  measurements  for  higher  accuracy 

Inspection  Efficiency: 

>  Automation  in  identifying  physical  damage  to  a  specific 
targeted  area  of  the  spacecraft 

>  Automation  in  reporting  dimensional  discrepancies  and 
out-of-tolerance  features 
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SPA  CECRAFT  IN-FLIGHT 
INSPECTION  SYSTEM 


\ 

Objectives 


Video  Example 
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CONCEPTUAL  DESIGN 


System  Overview 


High  Resolution 
2D  and  3D  Imaging 
Sensors 


7 


Possible  Untethered 
Autonomous  System 
(Mini  AERCam) 


Ground 

Processing 

System 


Provisional  Patent  Application  Filed 
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CONCEPTUAL  DESIGN 

(GROUND  PROCESSING  SYSTEM) 


System  Overview 


2D/3D  Front-end  User  Interface  Application  &  Controls 


2D/3D 

Image  Analysis 


New  SDK 

Middleware  ,  BioAPI,  SDK’s 

CAD  /  Baseline 

Integration 

CORE  TECHNOLOGY 


Exception 

Reporting 


2D/3D  Image 
Capture  / 
Reconstruction 


Robotic 
Camera 
Positioning  / 
Location 
Controls 


2D/3D 

Camera 

Configuration 

Controls 


2D/3D  Image 
Quality  Control 


Core  3D 

Core  2D 

Image  Capture  & 

Image  Capture  & 

Reconstruction 

Reconstruction 

Algorithm 

Algorithm 

Core  2D/3D 
Object  Analysis 
Algorithm 


Legend 


New  App 


Existing 

Technology 


{ 


Captured 

CAD  /  Baseline 

2D/3D 

Image 

Images 

DB 

Provisional  Patent  Application  Filed 
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CONCEPTUAL  DESIGN 


System  Overview 


AIS  Application  Software  Architect  tire  Design 
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System  Overview 


HOW  IS  OUR  SOLUTION  DIFFERENT? 


Combines  2D  &  3D  Imaging  Technologies 

>  Modular  Sensor  Design  (flexibility) 

>  3D  -  Higher  level  of  detail 

•  Near  Infrared  Technology 

•  Structured  Light 

>  5D  =  3D  with  Texture  for  human  eye  viewing 

Integrate  with  existing  COTS  CAD  vendors 

Higher  accuracy  in  detecting  dimensional  discrepancies 


Faster  reporting  of  damaged  parts  through  automation  of 
data  analytics 
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System  Overview 


2D  +  3D  =  5D 
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\ 

Key  Technology 


COMPONENTS  TO  SUCCESS 


Core  Technology: 

>  Combination  of  high  resolution 
imaging  sensors  to  make 
5D  model 

>  Stitching  algorithm  of 
synchronized  2D  and  3D 
image  capture  (us  Patent 

#6,694,064) 

>  Systems  integration  with 
COTS  CAD  products 

>  Automation  of  live  image  and 
stored  image  dimension 
comparison,  analysis  and  reporting 
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COMPONENTS  TO  SUCCESS 


\ 

Key  Technology 


Technology  Development: 


> 

> 


Single 

Camera 


LED  Array 


Dual 

Cameras 


Power  BUUn 
ciua« 


Single  camera 
Cluster 


Dun  cam  ?ro 
Cluster 

Avioiia  Ewara 
GSE  Port 


TcpHeiflJttisra 
GJTO  PBCR3B& 
Refuel  Cluster 


cen  L&r  strudirai  Rin 


QP&fieonwr 


Ed  lent  Hemrspnsre 


LECArr^'SWifld 


Transceiver  Pawags 


Mechanical  Integration  Design  of  Core  Technology  with  an  AIS 
Mechanical  Development  of  5D  Sensor 


MOUNTAIN  STATE 


INFORMATION  SYSTEMS,  INC. 


Copyright  ©2010,  Mountain  State  Information  Systems,  Inc.  All  rights  reserved.  -16- 


COMPONENTS  TO  SUCCESS 


\ 

Key  Technology 


Technology  Development: 

>  Software  Integration  with 
existing  COTS  CAD 

>  Image  Data  Analytics  and 
Reporting 

>  SDK  Development 

Key  Technical  Expertise: 

>  3D  Imaging  Systems 

■  Structured  Light 

■  Near  Infrared  Light  Source 


>  2D  and  3D  Image  Stitching 

>  System  Architecture  and  Integration  Design 
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BRIDGES  AND  MAJOR 
STRUCTURES 


;  \ 

Potential  Commercial 

Applications 


Current  Inspection: 

*  V  ; 

>  Treacherous 

>  Manual  Photos 

>  Expensive 

I  * 

Equipment 

jif 

Impact  to  Quality  of  Inspection: 

>  Vision  of  Inspector 

>  Experience 

>  Wellness  of 
Inspector 
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BRIDGES  AND  MAJOR 
STRUCTURES 


;  \ 

Potential  Commercial 

Applications 


❖  DraganFlyer™X6*: 

>  6  Rotor  Design 

>  Camera  Mounting  Brackets 

>  Counter  Rotating  Blades  for 
Stabilizers 


*  Draganfly  Innovations  Inc. 
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AIRCRAFT  FA  TIGUE  / 
DAMAGE 


;  \ 

Potential  Commercial 

Applications 


Current  Inspection: 

>  Visual  Inspection 

>  Using  sense  of  touch  of  the  hand 

>  Mirrors,  optical  micrometers 

Impact  to  Quality  of  Inspection: 

>  Vision  of  Inspector 

>  Experience 

>  Wellness  of 
Inspector 


Arrows  indicate  crack 
in  attach  angle 
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;  \ 

Potential  Applications 


MILITARY  APPLICA  TIONS 


•>  Scan  Roads  for  lEDs 


>  Provide  additional  solution,  either  better  than  current  solutions  or 
used  to  augment  current  solutions 

>  Potential  Platforms  (Mini  Hovering  UAV,  Fixed  Wing  UAVs) 

>  Take  images  of  roadways 

■  Compare  today  vs.  yesterday 

■  May  not  be  applicable  to  all  scenarios  (city  [clutter]  vs.  dirt  road  in 
unpopulated  area) 

Search  buildings,  vehicles  for  lEDs  (Ground  Robotics) 


Generate  3D  representations  of  interior  and  exterior 
spaces 
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PHASE  I 


\ 

Go  Forward 
Strategy 


Business  /  Analysis 

>  Pursue  other  government  opportunities 

>  Continue  due  diligence  on  previous  AIS  efforts 

>  NASA  Return  to  Flight  Task  Group  Final  Report  -  Resolution 
of  CAIB  recommendations:  Possible  additional  follow-up 

■  Update  of  CAD  Data  Base  -  satisfactory 

■  Automation  -  incomplete 

■  “Continued  vigilance  urged...” 


>  Continue  with  2D  and  3D  Best-of-Breed  technology  evaluation 

>  Continue  with  COTS  CAD  evaluations 

>  Develop  Sub-System  marketability  business  case  for 
commercial  applications 
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PHASE  I  (CONT.) 


\ 

Go  Forward 
Strategy 


Development 


> 

> 

> 


Complete  System  Integration  Design 
Complete  High-level  Software  Architecture 
Development  of  Tripod  Prototype: 


■  Develop  2D  and  3D  imaging  sensors 

■  Develop  algorithm  for  3D  image  capture,  stitching,  & 
reconstruction 

■  Develop  Synchronized  Stitching  Algorithm  of  2D  and  3D  images 

■  Test 
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CURRENT  EFFORTS 


\ 

Autonomous 
Inspection  System 


Effort  started  in  July  ’09 

Provisional  Patent  Development  &  Filing 

Establish  Strategic  Relationships  with  Initial  Customer 

Determine  funding  source(s) 

Submitted  informal  proposal  to  NASA 

Discussions  held  and  abstract  submitted  to  an  NSWC 
Dahlgren  division 
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TAKEAWAY  MESSAGES 


\ 

Autonomous 
Inspection  System 


❖  Technologies  Exist: 

>  Host  Platforms  are  available 

>  High  resolution  2D  and  3D  imaging  systems  are  available 

❖  Our  System 

>  Integrated  2D  /  3D  Imaging  System  to  capture  high  res  5D  images 

•  Near  Infrared 

•  Structured  Light 

•  2D  and  3D  Image  Stitching 

>  To  integrate  with  existing  commercial  COTS  CAD  systems 

>  Automated  image  inspection  to  compare  As-is  (live)  and  Baseline  data 

❖  Goal:  Provide  fast,  high  resolution  inspections 


Paul  Garnett 
970  708-7723 
pgarnett@msisinc.  com 
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Cody  Benkelman 
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